AMENDMENT UNDER 37 C.F.R. §1.111 
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AMENDMENTS TO THE CLAIMS 

This listing of claims will replace all prior versions and listings of claims in the 
application: 

LISTING OF CLAIMS: 

1. (currently amended): A method for synthesizing moving images, comprising the steps 

of: 

sampling two consecutive frames from a moving image; 

providing a reference patch having at least one rectangular region therein, in one 
of the two frames, which is designated as a reference frame; 

providing a second patch, which is the same as the reference patch, in the other of 
the two frames; 

moving and/or deforming the second patch in the other of the two frames so that 
an image within the second patch matches that within the reference image; 

estimating correspondent relationships among the-pixels within the second patch 
of the other frame and the-pixels within the reference patch of the reference frame, based on the 
second patch after the movement and/or deformation thereof and the reference patch; 

obtaining a coordinate converted frame, by coordinate converting the image 
within the second patch into a coordinate space of the reference frame, based on the 
correspondent relationships; 
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calculating a correlative value, which represents the-a_degree of correlation 
between the images within the reference patch of the reference frame and the coordinate 
converted frame; 

varying a-the number of said at least one rectangular regions within the reference 
patch and the second patch in a stepwise manner, estimating the aforem e ntioned correspondent 
relationships, obtaining the coordinate converted frames, and calculating the correlative values 
for each number of said at least one rectangular regions, thereby obtaining a plurality of 
correlative values corresponding to the number of rectangular regions within the reference patch 
and the second patch; 

comparing the degrees of correlation for each number of said at least one 
rectangular regions, based on the plurality of the correlative values; and 

generating a synthesized frame having a higher resolution than either of the two 
frames, by administering interpolation calculations on the images within the second patch and 
the reference patch, based on the correspondent relationships, which were estimated among the 
pixels of the reference patch and the second patch having the number of said at least one 
rectangular regions therein that yielded the highest degree of correlation. 

2. (currently amended): A method for synthesizing moving images, comprising the steps 

of: 

sampling at least three consecutive frames from a moving image; 
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providing a reference patch having at least one rectangular region therein, in a 
first frame from among the at least three sampled frames, which is designated as a reference 
frame; 

providing a second patch, which is the same as the reference patch, in another of 
the plurality of sampled frames; 

moving and/or deforming the second patch in the other frame so that an image 
within the second patch matches that within the reference image; 

estimating correspondent relationships among the-pixels within the second patch 
of the other frame and the-pixels within the reference patch of the reference frame, based on the 
second patch after the movement and/or deformation thereof and the reference patch; 

obtaining a coordinate converted frame, by coordinate converting the image 
within the second patch into a coordinate space of the reference frame, based on the 
correspondent relationships; 

calculating a correlative value, which represents the-adegree of correlation 
between the images within the reference patch of the reference frame and the coordinate 
converted frame; 

varying the number of said at least one rectangular regions within the reference 
patch and the second patch in a stepwise manner, estimating the afor e m e ntioned correspondent 
relationships, obtaining the coordinate converted frames, and calculating the correlative values 
for each number of said at least one rectangular regions, thereby obtaining a plurality of 
correlative values corresponding to the number of said at least one rectangular regions within the 
reference patch and the second patch; 
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comparing the degrees of correlation for each number of said at least one 
rectangular regions, based on the plurality of the correlative values; 

obtaining an intermediate synthesized frame having a higher resolution than either 
the reference frame of the other frame, by administering interpolation calculations on the images 
within the second patch and the reference patch, based on the correspondent relationships, which 
were estimated among the pixels of the reference patch and the second patch having the number 
of said at least one rectangular regions therein that yielded the highest degree of correlation; 

obtaining a plurality of intermediate synthesized frames by estimating the afor e mention e d 
correspondent relationships, obtaining the coordinate converted frames, calculating the plurality 
of correlative values, and comparing the degrees of correlation for all of the plurality of sampled 
frames; and 

generating a synthesized frame by synthesizing the plurality of intermediate synthesized 

frames. 

3. (currently amended): A moving image synthesizing apparatus, comprising: 

a sampling means, for sampling two consecutive frames from a moving image; 
a correspondent relationship estimating means, for: providing a reference patch 
having at least one rectangular region therein, in one of the two frames, which is designated as a 

reference frame; .providing a second patch, which is the same as the reference patch, in the 

other of the two frames; moving and/or deforming the second patch in the other of the two 
frames so that an image within the second patch matches that within the reference image; and 
estimating correspondent relationships among the-pixels within the second patch of the other 
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frame and tbe-pixels within the reference patch of the reference frame, based on the second patch 
after the movement and/or deformation thereof and the reference patch; 

a coordinate converting means, for obtaining a coordinate converted frame, by 
coordinate converting the image within the second patch into a coordinate space of the reference 
frame, based on the correspondent relationships; 

a correlative value calculating means, for calculating a correlative value, which 
represents the-a_degree of correlation between the images within the reference patch of the 
reference frame and the coordinate converted frame; 

a control means, for controlling the correspondent relationship estimating means, 
the coordinate converting means, and the correlative value calculating means, to vary the number 
of said at least one rectangular regions within the reference patch and the second patch in a 
stepwise manner, to estimate the afor e m e ntion e d correspondent relationships, to obtain the 
coordinate converted frames, and to calculate the correlative values for each number of said at 
least one rectangular regions, thereby obtaining a plurality of correlative values corresponding to 
the number of said at least one rectangular regions within the reference patch and the second 
patch; 

a comparing means, for comparing the degrees of correlation for each number of 
said at least one rectangular regions, based on the plurality of the correlative values; and 

a synthesizing means, for generating a synthesized frame having a higher 
resolution than either of the two frames, by administering interpolation calculations on the 
images within the second patch and the reference patch, based on the correspondent 
relationships, which were estimated among the pixels of the reference patch and the second patch 
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having the number of said at least one rectangular regions therein that yielded the highest degree 
of correlation. 

4. (original): A moving image synthesizing apparatus as defined in claim 3, wherein: 

the correspondent relationships employed in the generation of the synthesized 
frame are those which are estimated for each of the rectangular regions. 

5. (original): A moving image synthesizing apparatus as defined in claim 3, wherein: 

the correspondent relationship estimating means, the coordinate converting 
means, the correlative value calculating means, and the synthesizing means employ at least one 
component that constitutes the frames during the estimation of the correspondent relationships, 
the obtainment of the coordinate converted frames, the calculation of the correlative values, and 
the generation of the synthesized frame. 

6. (currently amended): A moving image synthesizing apparatus, comprising: 

a sampling means, for sampling at least three consecutive frames from a moving 

image; 

a correspondent relationship estimating means, for: providing a reference patch 
having at least one rectangular region therein, in a first frame from among the at least three 
sampled frames, which is designated as a reference frame; providing a second patch, which is the 
same as the reference patch, in another of the plurality of sampled frames; moving and/or 
deforming the second patch in the other frame so that an image within the second patch matches 
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that within the reference image; and estimating correspondent relationships among-the pixels 
within the second patch of the other frame and the-pixels within the reference patch of the 
reference frame, based on the second patch after the movement and/or deformation thereof and 
the reference patch; 

a coordinate converting means, for obtaining a coordinate converted frame, by 
coordinate converting the image within the second patch into a coordinate space of the reference 
frame, based on the correspondent relationships; 

a correlative value calculating means, for calculating a correlative value, which 
represents the-a_degree of correlation between the images within the reference patch of the 
reference frame and the coordinate converted frame; 

a control means, for controlling the correspondent relationship estimating means, 
the coordinate converting means, and the correlative value calculating means, to vary the number 
of said at least one rectangular regions within the reference patch and the second patch in a 
stepwise manner, to estimate the afor e m e ntioned correspondent relationships, to obtain the 
coordinate converted frames, and to calculate the correlative values for each number of said at 
least one rectangular regions, thereby obtaining a plurality of correlative values corresponding to 
the number of said at least one rectangular regions within the reference patch and the second 
patch; 

a comparing means, for comparing the degrees of correlation for each number of 
said at least one rectangular regions, based on the plurality of the correlative values; and 

a synthesizing means, for: obtaining an intermediate synthesized frame having a 
higher resolution than either the reference frame of the other frame, by administering 
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interpolation calculations on the images within the second patch and the reference patch, based 
on the correspondent relationships, which were estimated among the pixels of the reference patch 
and the second patch having the number of said at least one rectangular regions therein that 
yielded the highest degree of correlation; obtaining a plurality of intermediate synthesized frames 
by estimating the afor e m e ntion e d correspondent relationships, obtaining the coordinate 
converted frames, calculating the plurality of correlative values, and comparing the correlations 
for all of the plurality of sampled frames; and generating a synthesized frame by synthesizing the 
plurality of intermediate synthesized frames. 

7. (original): A moving image synthesizing apparatus as defined in claim 6, wherein: 

the correspondent relationships employed in the generation of the intermediate 
synthesized frame are those which are estimated for each of the rectangular regions. 

8. (original): A moving image synthesizing apparatus as defined in claim 6, wherein: 

the correspondent relationship estimating means, the coordinate converting 
means, the correlative value calculating means, and the synthesizing means employ at least one 
component that constitutes the frames during the estimation of the correspondent relationships, 
the obtainment of the coordinate converted frames, the calculation of the correlative values, the 
generation of the intermediate synthesized frames, and the generation of the synthesized frame. 
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9. (currently amended): A recording medium having embodied thereon A a program that 
causes a computer to execute a method for synthesizing moving images, comprising the 
procedures of: 

sampling two consecutive frames from a moving image; 

providing a reference patch having at least one rectangular region therein, in one 
of the two frames, which is designated as a reference frame; 

providing a second patch, which is the same as the reference patch, in the other of 
the two frames; 

moving and/or deforming the second patch in the other of the two frames so that 
an image within the second patch matches that within the reference image; 

estimating correspondent relationships among the-pixels within the second patch 
of the other frame and the-pixels within the reference patch of the reference frame, based on the 
second patch after the movement and/or deformation thereof and the reference patch; 

obtaining a coordinate converted frame, by coordinate converting the image 
within the second patch into a coordinate space of the reference frame, based on the 
correspondent relationships; 

calculating a correlative value, which represents the-ajiegree of correlation 
between the images within the reference patch of the reference frame and the coordinate 
converted frame; 

varying the number of said at least one rectangular regions within the reference 
patch and the second patch in a stepwise manner, estimating the afor e m e ntion e d correspondent 
relationships, obtaining the coordinate converted frames, and calculating the correlative values 
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for each number of said at least one rectangular regions, thereby obtaining a plurality of 
correlative values corresponding to the number of said at least one rectangular regions within the 
reference patch and the second patch; 

comparing the degrees of correlation for each number of said at least one 
rectangular regions, based on the plurality of the correlative values; and 

generating a synthesized frame having a higher resolution than either of the two 
frames, by administering interpolation calculations on the images within the second patch and 
the reference patch, based on the correspondent relationships, which were estimated among the 
pixels of the reference patch and the second patch having the number of said at least one 
rectangular regions therein that yielded the highest degree of correlation. 

10. (currently amended): A recording medium having embodied thereon Aa program 
that causes a computer to execute a method for synthesizing moving images, comprising the 
procedures of: 

sampling at least three consecutive frames from a moving image; 

providing a reference patch having at least one rectangular region therein, in a 
first frame from among the at least three sampled frames, which is designated as a reference 
frame; 

providing a second patch, which is the same as the reference patch, in another of 
the plurality of sampled frames; 

moving and/or deforming the second patch in the other frame so that an image 
within the second patch matches that within the reference image; 
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estimating correspondent relationships among the-pixels within the second patch 
of the other frame and the-pixels within the reference patch of the reference frame, based on the 
second patch after the movement and/or deformation thereof and the reference patch; 

obtaining a coordinate converted frame, by coordinate converting the image 
within the second patch into a coordinate space of the reference frame, based on the 
correspondent relationships; 

calculating a correlative value, which represents the-a_degree of correlation 
between the images within the reference patch of the reference frame and the coordinate 
converted frame; 

varying the number of said at least one rectangular regions within the reference 
patch and the second patch in a stepwise manner, estimating the aforem e ntioned correspondent 
relationships, obtaining the coordinate converted frames, and calculating the correlative values 
for each number of said at least one rectangular regions, thereby obtaining a plurality of 
correlative values corresponding to the number of said at least one rectangular regions within the 
reference patch and the second patch; 

comparing the degrees of correlation for each number of said at least one 
rectangular regions, based on the plurality of the correlative values; 

obtaining an intermediate synthesized frame having a higher resolution than either 
the reference frame of the other frame, by administering interpolation calculations on the images 
within the second patch and the reference patch, based on the correspondent relationships, which 
were estimated among the pixels of the reference patch and the second patch having the number 
of said at least one rectangular regions therein that yielded the highest degree of correlation; 
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obtaining a plurality of intermediate synthesized frames by estimating the aforementioned 
correspondent relationships, obtaining the coordinate converted frames, calculating the plurality 
of correlative values, and comparing the degrees of correlation for all of the plurality of sampled 
frames; and 

generating a synthesized frame by synthesizing the plurality of intermediate synthesized 

frames. 

11. (currently amended): A method for synthesizing moving images, comprising the 
steps of: 

sampling two consecutive frames from a moving image; 

providing a reference patch having at least one rectangular region therein, in one 
of the two frames, which is designated as a reference frame; 

providing a second patch, which is the same as the reference patch, in the other of 
the two frames; 

moving and/or deforming the second patch in the other of the two frames so that 
an image within the second patch matches that within the reference image; 

estimating correspondent relationships among the-pixels within the second patch 
of the other frame and the-pixels within the reference patch of the reference frame, based on the 
second patch after the movement and/or deformation thereof and the reference patch; 

obtaining a coordinate converted frame, by coordinate converting the image 
within the second patch into a coordinate space of the reference frame, based on the 
correspondent relationships; 
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calculating a correlative value, which represents the-a_degree of correlation 
between the images within the reference patch of the reference frame and the coordinate 
converted frame; 

increasing the number of said at least one rectangular regions within the reference 
patch and the second patch in a stepwise manner, estimating the afor e mention e d correspondent 
relationships, obtaining the coordinate converted frames, and calculating the correlative values 
for each number of said at least one rectangular regions, until the degree of correlation is greater 
than or equal to a predetermined threshold value; and 

generating a synthesized frame having a higher resolution than either of the two 
frames, by administering interpolation calculations on the images within the second patch and 
the reference patch, based on the correspondent relationships, which were estimated among the 
pixels of the reference patch and the second patch having the number of said at least one 
rectangular regions therein that yielded the degree of correlation greater than or equal to the 
threshold value. 

12. (currently amended): A method for synthesizing moving images, comprising the 
steps of: 

sampling two consecutive frames from a moving image; 

providing a reference patch having at least one rectangular region therein, in one 
of the two frames, which is designated as a reference frame; 

providing a second patch, which is the same as the reference patch, in the other of 
the two frames; 

14 



AMENDMENT UNDER 37 C.F.R. §1.111 
Application No.: 10/750,461 



Attorney Docket No.: Q79263 



moving and/or deforming the second patch in the other of the two frames so that 
an image within the second patch matches that within the reference image; 

estimating correspondent relationships among the-pixels within the second patch 
of the other frame and the-pixels within the reference patch of the reference frame, based on the 
second patch after the movement and/or deformation thereof and the reference patch; 

obtaining a coordinate converted frame, by coordinate converting the image 
within the second patch into a coordinate space of the reference frame, based on the 
correspondent relationships; 

calculating correlative values, which represent the-a_degree of correlation between 
the images within the reference patch of the reference frame and the coordinate converted frame, 
for each of the rectangular regions; 

increasing the number of said at least one rectangular regions within the reference 
patch and the second patch, in a stepwise manner, estimating the afor e mention e d correspondent 
relationships, obtaining the coordinate converted frames, and calculating the correlative values 
for each number of said at least one rectangular regions, until the degree of correlation or a 
representative degree of correlation of all of the- said at least one rectangular regions, or of said at 
least one rectangular regions at predetermined positions from among all of the- said at least one 
rectangular regions, other than those for which the degree of correlation is less than a 
predetermined value, or of a predetermined number of said at least one rectangular regions from 
among all of the- said at least one rectangular regions, other than those for which the degree of 
correlation is less than a predetermined value, becomes greater than or equal to a predetermined 
threshold value; and 
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generating a synthesized frame having a higher resolution than either of the two 
frames, by administering interpolation calculations on the images within the second patch and 
the reference patch for each region corresponding to the- said at least one rectangular regions, 
based on the correspondent relationships, which were estimated among the pixels of the 
reference patch and the second patch having the number of said at least one rectangular regions 
therein that yielded the degree of correlation or the representative degree of correlation greater 
than or equal to the threshold value. 

13. (currently amended): A method for synthesizing moving images, comprising 
the steps of: 

sampling at least three consecutive frames from a moving image; 

providing a reference patch having at least one rectangular region therein, in a 
first frame from among the at least three sampled frames, which is designated as a reference 
frame; 

providing a second patch, which is the same as the reference patch, in another of 
the plurality of sampled frames; 

moving and/or deforming the second patch in the other frame so that an image 
within the second patch matches that within the reference image; 

estimating correspondent relationships among the-pixels within the second patch 
of the other frame and the-pixels within the reference patch of the reference frame, based on the 
second patch after the movement and/or deformation thereof and the reference patch; 
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obtaining a coordinate converted frame, by coordinate converting the image 
within the second patch into a coordinate space of the reference frame, based on the 
correspondent relationships; 

calculating a correlative value, which represents the-a_degree of correlation 
between the images within the reference patch of the reference frame and the coordinate 
converted frame; 

increasing the number of said at least one rectangular regions within the reference 
patch and the second patch in a stepwise manner, estimating the afor e m e ntioned correspondent 
relationships, obtaining the coordinate converted frames, and calculating the correlative values 
for each number of said at least one rectangular regions, until the degree of correlation is greater 
than or equal to a predetermined threshold value; 

obtaining an intermediate synthesized frame having a higher resolution than either 
the reference frame of the other frame, by administering interpolation calculations on the images 
within the second patch and the reference patch for each region corresponding to the- said at least 
one rectangular regions, based on the correspondent relationships, which were estimated among 
the pixels of the reference patch and the second patch having the number of said at least one 
rectangular regions therein that yielded the degree of correlation greater than or equal to the 
threshold value; 

obtaining a plurality of intermediate synthesized frames by estimating the aforem e ntion e d 
correspondent relationships, obtaining the coordinate converted frames, and calculating the 
plurality of correlative values, for all of the plurality of sampled frames; and 
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generating a synthesized frame by synthesizing the plurality of intermediate synthesized 

frames. 

14. (currently amended): A method for synthesizing moving images, comprising the 
steps of: 

sampling at least three consecutive frames from a moving image; 

providing a reference patch having at least one rectangular region therein, in a 
first frame from among the at least three sampled frames, which is designated as a reference 
frame; 

providing a second patch, which is the same as the reference patch, in another of 
the plurality of sampled frames; 

moving and/or deforming the second patch in the other frame so that an image 
within the second patch matches that within the reference image; 

estimating correspondent relationships among the-pixels within the second patch 
of the other frame and the-pixels within the reference patch of the reference frame, based on the 
second patch after the movement and/or deformation thereof and the reference patch; 

obtaining a coordinate converted frame, by coordinate converting the image 
within the second patch into a coordinate space of the reference frame, based on the 
correspondent relationships; 

calculating a correlative value, which represents the-adegree of correlation 
between the images within the reference patch of the reference frame and the coordinate 
converted frame; 
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increasing the number of said at least one rectangular regions within the reference 
patch and the second patch in a stepwise manner, estimating the aforem e ntion e d correspondent 
relationships, obtaining the coordinate converted frames, and calculating the correlative values 
for each number of said at least one rectangular regions, until the degree of correlation or a 
representative degree of correlation of all of said at least one rectangular regions, or of said at 
least one rectangular regions at predetermined positions from among all of the- said at least one 
rectangular regions, other than those for which the degree of correlation is less than a 
predetermined value, or of a predetermined number of said at least one rectangular regions from 
among all of the- said at least one rectangular regions, other than those for which the degree of 
correlation is less than a predetermined value, becomes greater than or equal to a predetermined 
threshold value; 

obtaining an intermediate synthesized frame having a higher resolution than either 
the reference frame of the other frame, by administering interpolation calculations on the images 
within the second patch and the reference patch for each region corresponding to the- said at least 
one rectangular regions, based on the correspondent relationships, which were estimated among 
the pixels of the reference patch and the second patch having the number of said at least one 
rectangular regions therein that yielded the degree of correlation or the representative degree of 
correlation greater than or equal to the threshold value; 

obtaining a plurality of intermediate synthesized frames by estimating the afor e mention e d 
correspondent relationships, obtaining the coordinate converted frames, and calculating the 
plurality of correlative values, for all of the plurality of sampled frames; and 
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generating a synthesized frame by synthesizing the plurality of intermediate synthesized 

frames. 

15. (currently amended): A moving image synthesizing apparatus comprising: 

a sampling means, for sampling two consecutive frames from a moving image; 

a correspondent relationship estimating means, for: providing a reference patch 
having at least one rectangular region therein, in one of the two frames, which is designated as a 

reference frame; .providing a second patch, which is the same as the reference patch, in the 

other of the two frames; moving and/or deforming the second patch in the other of the two 
frames so that an image within the second patch matches that within the reference image; and 
estimating correspondent relationships among the-pixels within the second patch of the other 
frame and the-pixels within the reference patch of the reference frame, based on the second patch 
after the movement and/or deformation thereof and the reference patch; 

a coordinate converting means, for obtaining a coordinate converted frame, by 
coordinate converting the image within the second patch into a coordinate space of the reference 
frame, based on the correspondent relationships; 

a correlative value calculating means, for calculating a correlative value, which 
represents the-a_degree of correlation between the images within the reference patch of the 
reference frame and the coordinate converted frame; 

a control means, for controlling the correspondent relationship estimating means, 
the coordinate converting means, and the correlative value calculating means, to increase the 
number of said at least one rectangular regions within the reference patch and the second patch in 
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a stepwise manner, to estimate the afor e m e ntion e d correspondent relationships, to obtain the 
coordinate converted frames, and to calculate the correlative values for each number of said at 
least one rectangular regions, until the degree of correlation is greater than or equal to a 
predetermined threshold value; and 

a synthesizing means, for generating a synthesized frame having a higher 
resolution than either of the two frames, by administering interpolation calculations on the 
images within the second patch and the reference patch, based on the correspondent 
relationships, which were estimated among the pixels of the reference patch and the second patch 
having the number of said at least one rectangular regions therein that yielded the degree of 
correlation greater than or equal to the predetermined threshold value. 

16. (currently amended): A moving image synthesizing apparatus comprising: 

a sampling means, for sampling two consecutive frames from a moving image; 
a correspondent relationship estimating means, for: providing a reference patch 
having at least one rectangular region therein, in one of the two frames, which is designated as a 

reference frame;, providing a second patch, which is the same as the reference patch, in the 

other of the two frames; moving and/or deforming the second patch in the other of the two 
frames so that an image within the second patch matches that within the reference image; and 
estimating correspondent relationships among the-pixels within the second patch of the other 
frame and the-pixels within the reference patch of the reference frame, based on the second patch 
after the movement and/or deformation thereof and the reference patch; 
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a coordinate converting means, for obtaining a coordinate converted frame, by 
coordinate converting the image within the second patch into a coordinate space of the reference 
frame, based on the correspondent relationships; 

a correlative value calculating means, for calculating a correlative value, which 
represents the-a.de gree of correlation between the images within the reference patch of the 
reference frame and the coordinate converted frame; 

a control means, for controlling the correspondent relationship estimating means, 
the coordinate converting means, and the correlative value calculating means, to increase the 
number of said at least one rectangular regions within the reference patch and the second patch in 
a stepwise manner, to estimate the afor e m e ntion e d correspondent relationships, to obtain the 
coordinate converted frames, and to calculate the correlative values for each number of said at 
least one rectangular regions, until the degree of correlation or a representative degree of 
correlation of all of said at least one the rectangular regions, or of said at least one rectangular 
regions at predetermined positions from among all of said at least one the rectangular regions, 
other than those for which the degree of correlation is less than a predetermined value, or of a 
predetermined number of said at least one rectangular regions from among all of said at least 
onethe rectangular regions, other than those for which the degree of correlation is less than a 
predetermined value, becomes greater than or equal to a predetermined threshold value; and 

a synthesizing means, for generating a synthesized frame having a higher 
resolution than either of the two frames, by administering interpolation calculations on the 
images within the second patch and the reference patch for each region corresponding to said at 
least one rectangular regions, based on the correspondent relationships, which were estimated 
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among the pixels of the reference patch and the second patch having the number of said at least 
one rectangular regions therein that yielded the degree of correlation or the representative degree 
of correlation greater than or equal to the predetermined threshold value. 

17. (original): A moving image synthesizing apparatus as defined in claim 15, wherein: 

the control means decreases the threshold value, according to the increase in the 
number of rectangular regions, until the number of rectangular regions reaches a predetermined 
value, and increases the threshold value, when the number of rectangular regions exceeds the 
predetermined value. 

18. (original): A moving image synthesizing apparatus as defined in claim 16, wherein: 

the control means decreases the threshold value, according to the increase in the 
number of rectangular regions, until the number of rectangular regions reaches a predetermined 
value, and increases the threshold value, when the number of rectangular regions exceeds the 
predetermined value. 

19. (original): A moving image synthesizing apparatus as defined in claim 15, wherein: 

the correspondent relationship estimating means, the coordinate converting 
means, the correlative value calculating means, and the synthesizing means employ at least one 
component that constitutes the frames during the estimation of the correspondent relationships, 
the obtainment of the coordinate converted frames, the calculation of the correlative values, and 
the generation of the synthesized frame. 
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20. (original): A moving image synthesizing apparatus as defined in claim 16, wherein: 

the correspondent relationship estimating means, the coordinate converting 
means, the correlative value calculating means, and the synthesizing means employ at least one 
component that constitutes the frames during the estimation of the correspondent relationships, 
the obtainment of the coordinate converted frames, the calculation of the correlative values, and 
the generation of the synthesized frame. 

21. (currently amended): A moving image synthesizing apparatus, comprising: 

a sampling means, for sampling at least three consecutive frames from a moving 

image; 

a correspondent relationship estimating means, for: providing a reference patch 
having at least one rectangular region therein, in a first frame from among the at least three 
sampled frames, which is designated as a reference frame; providing a second patch, which is the 
same as the reference patch, in another of the plurality of sampled frames; moving and/or 
deforming the second patch in the other frame so that an image within the second patch matches 
that within the reference image; and estimating correspondent relationships among the-pixels 
within the second patch of the other frame and the-pixels within the reference patch of the 
reference frame, based on the second patch after the movement and/or deformation thereof and 
the reference patch; 
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a coordinate converting means, for obtaining a coordinate converted frame, by 
coordinate converting the image within the second patch into a coordinate space of the reference 
frame, based on the correspondent relationships; 

a correlative value calculating means, for calculating a correlative value, which 
represents the-ajiegree of correlation between the images within the reference patch of the 
reference frame and the coordinate converted frame; 

a control means, for controlling the correspondent relationship estimating means, 
the coordinate converting means, and the correlative value calculating means, to increase the 
number of said at least one rectangular regions within the reference patch and the second patch in 
a stepwise manner, to estimate the afor e m e ntion e d correspondent relationships, to obtain the 
coordinate converted frames, and to calculate the correlative values for each number of said at 
least one rectangular regions, until the degree of correlation is greater than or equal to a 
predetermined threshold value; and 

a synthesizing means, for: obtaining an intermediate synthesized frame having a 
higher resolution than either the reference frame of the other frame, by: administering 
interpolation calculations on the images within the second patch and the reference patch, based 
on the correspondent relationships, which were estimated among the pixels of the reference patch 
and the second patch having the number of said at least one rectangular regions therein that 
yielded the degree of correlation greater than or equal to the threshold value; obtaining a 
plurality of intermediate synthesized frames by estimating the aforem e ntion e d correspondent 
relationships obtaining the coordinate converted frames, and calculating the plurality of 
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correlative values, for all of the plurality of sampled frames; and generating a synthesized frame 
by synthesizing the plurality of intermediate synthesized frames. 

22. (currently amended): A moving image synthesizing apparatus, comprising: 

a sampling means, for sampling at least three consecutive frames from a moving 

image; 

a correspondent relationship estimating means, for: providing a reference patch 
having at least one rectangular region therein, in a first frame from among the at least three 
sampled frames, which is designated as a reference frame; providing a second patch, which is the 
same as the reference patch, in another of the plurality of sampled frames; moving and/or 
deforming the second patch in the other frame so that an image within the second patch matches 
that within the reference image; and estimating correspondent relationships among the-pixels 
within the second patch of the other frame and the-pixels within the reference patch of the 
reference frame, based on the second patch after the movement and/or deformation thereof and 
the reference patch; 

a coordinate converting means, for obtaining a coordinate converted frame, by 
coordinate converting the image within the second patch into a coordinate space of the reference 
frame, based on the correspondent relationships; 

a correlative value calculating means, for calculating a correlative value, which 
represents the-a_degree of correlation between the images within the reference patch of the 
reference frame and the coordinate converted frame; 
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a control means, for controlling the correspondent relationship estimating means, 
the coordinate converting means, and the correlative value calculating means, to increase the 
number of said at least one rectangular regions within the reference patch and the second patch in 
a stepwise manner, to estimate the afor e m e ntion e d correspondent relationships, to obtain the 
coordinate converted frames, and to calculate the correlative values for each number of said at 
least one rectangular regions, until the degree of correlation or a representative degree of 
correlation of all of said at least onet he rectangular regions, or of said at least one rectangular 
regions at predetermined positions from among all of said at least one the rectangular regions, 
other than those for which the degree of correlation is less than a predetermined value, or of a 
predetermined number of said at least one rectangular regions from among all of said at least 
onethe rectangular regions, other than those for which the degree of correlation is less than a 
predetermined value, becomes greater than or equal to a predetermined threshold value; and 

a synthesizing means, for: obtaining an intermediate synthesized frame having a 
higher resolution than either the reference frame of the other frame, by: administering 
interpolation calculations on the images within the second patch and the reference patch for each 
region corresponding to said at least one the-rectangular regions, based on the correspondent 
relationships, which were estimated among the pixels of the reference patch and the second patch 
having the number of said at least one rectangular regions therein that yielded the degree of 
correlation or the representative degree of correlation greater than or equal to the threshold value; 
obtaining a plurality of intermediate synthesized frames by estimating the afor e m e ntion e d 
correspondent relationships obtaining the coordinate converted frames, and calculating the 
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plurality of correlative values, for all of the plurality of sampled frames; and generating a 
synthesized frame by synthesizing the plurality of intermediate synthesized frames. 

23. (original): A moving image synthesizing apparatus as defined in claim 21, wherein: 

the control means decreases the threshold value, according to the increase in the 
number of rectangular regions, until the number of rectangular regions reaches a predetermined 
value, and increases the threshold value, according to the increase in the number of rectangular 
regions, when the number of rectangular regions exceeds the predetermined value. 

24. (original): A moving image synthesizing apparatus as defined in claim 22, wherein: 

the control means decreases the threshold value, according to the increase in the 
number of rectangular regions, until the number of rectangular regions reaches a predetermined 
value, and increases the threshold value, according to the increase in the number of rectangular 
regions, when the number of rectangular regions exceeds the predetermined value. 

25. (original): A moving image synthesizing apparatus as defined in claim 21, wherein: 

the correspondent relationship estimating means, the coordinate converting 
means, the correlative value calculating means, and the synthesizing means employ at least one 
component that constitutes the frames during the estimation of the correspondent relationships, 
the obtainment of the coordinate converted frames, the calculation of the correlative values, the 
generation of the intermediate synthesized frames, and the generation of the synthesized frame. 
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26. (original): A moving image synthesizing apparatus as defined in claim 22, wherein: 

the correspondent relationship estimating means, the coordinate converting 
means, the correlative value calculating means, and the synthesizing means employ at least one 
component that constitutes the frames during the estimation of the correspondent relationships, 
the obtainment of the coordinate converted frames, the calculation of the correlative values, the 
generation of the intermediate synthesized frames, and the generation of the synthesized frame. 

27. (currently amended): A recording medium having embodied thereon Aa program 
that causes a computer to execute a method for synthesizing moving images, comprising the 
procedures of: 

sampling two consecutive frames from a moving image; 

providing a reference patch having at least one rectangular region therein, in one 
of the two frames, which is designated as a reference frame; 

providing a second patch, which is the same as the reference patch, in the other of 
the two frames; 

moving and/or deforming the second patch in the other of the two frames so that 
an image within the second patch matches that within the reference image; 

estimating correspondent relationships among the-pixels within the second patch 
of the other frame and the-pixels within the reference patch of the reference frame, based on the 
second patch after the movement and/or deformation thereof and the reference patch; 
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obtaining a coordinate converted frame, by coordinate converting the image 
within the second patch into a coordinate space of the reference frame, based on the 
correspondent relationships; 

calculating a correlative value, which represents the-ajiegree of correlation 
between the images within the reference patch of the reference frame and the coordinate 
converted frame; 

increasing the number of said at least one rectangular regions within the reference 
patch and the second patch in a stepwise manner, estimating the afor e m e ntion e d correspondent 
relationships, obtaining the coordinate converted frames, and calculating the correlative values 
for each number of said at least one rectangular regions, until the degree of correlation is greater 
than or equal to a predetermined threshold value; and 

generating a synthesized frame having a higher resolution than either of the two 
frames, by administering interpolation calculations on the images within the second patch and 
the reference patch, based on the correspondent relationships, which were estimated among the 
pixels of the reference patch and the second patch having the number of said at least one 
rectangular regions therein that yielded the degree of correlation greater than or equal to the 
threshold value. 

28. (currently amended): A recording medium having embodied thereon Aa program 
that causes a computer to execute a method for synthesizing moving images, comprising the 
procedures of: 

sampling two consecutive frames from a moving image; 
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providing a reference patch having at least one rectangular region therein, in one 
of the two frames, which is designated as a reference frame; 

providing a second patch, which is the same as the reference patch, in the other of 
the two frames; 

moving and/or deforming the second patch in the other of the two frames so that 
an image within the second patch matches that within the reference image; 

estimating correspondent relationships among the-pixels within the second patch 
of the other frame and the-pixels within the reference patch of the reference frame, based on the 
second patch after the movement and/or deformation thereof and the reference patch; 

obtaining a coordinate converted frame, by coordinate converting the image 
within the second patch into a coordinate space of the reference frame, based on the 
correspondent relationships; 

calculating correlative values, which represent the-a_degree of correlation between 
the images within the reference patch of the reference frame and the coordinate converted frame, 
for each of the rectangular regions; 

increasing the number of said at least one rectangular regions within the reference 
patch and the second patch, in a stepwise manner, estimating the afor e m e ntion e d correspondent 
relationships, obtaining the coordinate converted frames, and calculating the correlative values 
for each number of said at least one rectangular regions, until the degree of correlation or a 
representative degree of correlation of all of said at least one the rectangular regions, or of said at 
least one rectangular regions at predetermined positions from among all of said at least one the 
rectangular regions, other than those for which the degree of correlation is less than a 
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predetermined value, or of a predetermined number of said at least one rectangular regions from 
among all of said at least one the-rectangular regions, other than those for which the degree of 
correlation is less than a predetermined value, becomes greater than or equal to a predetermined 
threshold value; and 

generating a synthesized frame having a higher resolution than either of the two 
frames, by administering interpolation calculations on the images within the second patch and 
the reference patch for each region corresponding to said at least one the rectangular regions, 
based on the correspondent relationships, which were estimated among the pixels of the 
reference patch and the second patch having the number of said at least one rectangular regions 
therein that yielded the degree of correlation or the representative degree of correlation greater 
than or equal to the threshold value. 

29. (currently amended): A recording medium having embodied thereon A a program 
that causes a computer to execute a method for synthesizing moving images, comprising the 
procedures of: 

sampling at least three consecutive frames from a moving image; 

providing a reference patch having at least one rectangular region therein, in a 
first frame from among the at least three sampled frames, which is designated as a reference 
frame; 

providing a second patch, which is the same as the reference patch, in another of 
the plurality of sampled frames; 
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moving and/or deforming the second patch in the other frame so that an image 
within the second patch matches that within the reference image; 

estimating correspondent relationships among tbe-pixels within the second patch 
of the other frame and the-pixels within the reference patch of the reference frame, based on the 
second patch after the movement and/or deformation thereof and the reference patch; 

obtaining a coordinate converted frame, by coordinate converting the image 
within the second patch into a coordinate space of the reference frame, based on the 
correspondent relationships; 

calculating a correlative value, which represents the-a^degree of correlation 
between the images within the reference patch of the reference frame and the coordinate 
converted frame; 

increasing the number of said at least one rectangular regions within the reference 
patch and the second patch in a stepwise manner, estimating the afor e mention e d correspondent 
relationships, obtaining the coordinate converted frames, and calculating the correlative values 
for each number of said at least one rectangular regions, until the degree of correlation is greater 
than or equal to a predetermined threshold value; 

obtaining an intermediate synthesized frame having a higher resolution than either 
the reference frame of the other frame, by administering interpolation calculations on the images 
within the second patch and the reference patch, based on the correspondent relationships, which 
were estimated among the pixels of the reference patch and the second patch having the number 
of said at least one rectangular regions therein that yielded the degree of correlation greater than 
or equal to the threshold value; 
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obtaining a plurality of intermediate synthesized frames by estimating the 
afor e m e ntioned correspondent relationships, obtaining the coordinate converted frames, and 
calculating the plurality of correlative values, for all of the plurality of sampled frames; and 

generating a synthesized frame by synthesizing the plurality of intermediate 
synthesized frames. 

30. (currently amended): A recording medium having embodied thereon Aa program 
that causes a computer to execute a method for synthesizing moving images, comprising the 
procedures of: 

sampling at least three consecutive frames from a moving image; 

providing a reference patch having at least one rectangular region therein, in a 
first frame from among the at least three sampled frames, which is designated as a reference 
frame; 

providing a second patch, which is the same as the reference patch, in another of 
the plurality of sampled frames; 

moving and/or deforming the second patch in the other frame so that an image 
within the second patch matches that within the reference image; 

estimating correspondent relationships among the-pixels within the second patch 
of the other frame and the-pixels within the reference patch of the reference frame, based on the 
second patch after the movement and/or deformation thereof and the reference patch; 
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obtaining a coordinate converted frame, by coordinate converting the image 
within the second patch into a coordinate space of the reference frame, based on the 
correspondent relationships; 

calculating a correlative value, which represents the-adegree of correlation 
between the images within the reference patch of the reference frame and the coordinate 
converted frame; 

increasing the number of said at least one rectangular regions within the reference 
patch and the second patch in a stepwise manner, estimating the afor e m e ntion e d correspondent 
relationships, obtaining the coordinate converted frames, and calculating the correlative values 
for each number of said at least one rectangular regions, until the degree of correlation or a 
representative degree of correlation of all of said at least one the-rectangular regions, or of said at 
least one rectangular regions at predetermined positions from among all of said at least one the 
rectangular regions, other than those for which the degree of correlation is less than a 
predetermined value, or of a predetermined number of said at least one rectangular regions from 
among all o f said at least one -fee rectangular regions, other than those for which the degree of 
correlation is less than a predetermined value, becomes greater than or equal to a predetermined 
threshold value; 

obtaining an intermediate synthesized frame having a higher resolution than either 
the reference frame of the other frame, by administering interpolation calculations on the images 
within the second patch and the reference patch for each region corresponding to said at least one 
rectangular regions, based on the correspondent relationships, which were estimated among the 
pixels of the reference patch and the second patch having the number of said at least one 
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rectangular regions therein that yielded the degree of correlation or the representative degree of 
correlation greater than or equal to the threshold value; 

obtaining a plurality of intermediate synthesized frames by estimating the 
afor e m e ntion e d correspondent relationships, obtaining the coordinate converted frames, and 
calculating the plurality of correlative values, for all of the plurality of sampled frames; and 

generating a synthesized frame by synthesizing the plurality of intermediate 
synthesized frames. 

31. (currently amended): A method for synthesizing moving images, comprising the 
steps of: 

sampling two consecutive frames from a moving image; 

providing a reference patch having at least one rectangular region therein, in one 
of the two frames, which is designated as a reference frame; 

providing a second patch, which is the same as the reference patch, in the other of 
the two frames; 

moving and/or deforming the second patch in the other of the two frames so that 
an image within the second patch matches that within the reference image; 

estimating correspondent relationships among the-pixels within the second patch 
of the other frame and the-pixels within the reference patch of the reference frame, based on the 
second patch after the movement and/or deformation thereof and the reference patch; 
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obtaining a coordinate converted frame, by coordinate converting the image 
within the second patch into a coordinate space of the reference frame, based on the 
correspondent relationships; 

calculating a first correlative value, which represents the-ajiegree of correlation 
between the images within the reference patch of the reference frame and the coordinate 
converted frame; 

increasing the number o f said at least one rectangular regions within the reference 
patch and the second patch in a stepwise manner by one step, estimating the aforem e ntion e d 
correspondent relationships, obtaining the coordinate converted frames, and calculating the 
correlative value, thereby obtaining a second correlative value; 

calculating the variation between the first and second correlative values; 

increasing the number of said at least one rectanRular regions within the reference 
patch and the second patch in a stepwise manner an additional step, estimating the 
afor e m e ntion e d correspondent relationships, obtaining the coordinate converted frames, 
calculating the first and second correlative values, and calculating the variation between the first 
and second correlative values of consecutive steps, until the variation becomes less than or equal 
to a predetermined threshold value; and 

generating a synthesized frame having a higher resolution than either of the two 
frames, by administering interpolation calculations on the images within the second patch and 
the reference patch, based on the correspondent relationships, which were estimated among the 
pixels of either of the reference patches and the second patches having the number of said at least 
one rectangular regions therein that yielded the variation less than or equal to the threshold value. 
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32. (currently amended): A method for synthesizing moving images, comprising the 
steps of: 

sampling two consecutive frames from a moving image; 

providing a reference patch having at least one rectangular region therein, in one 
of the two frames, which is designated as a reference frame; 

providing a second patch, which is the same as the reference patch, in the other of 
the two frames; 

moving and/or deforming the second patch in the other of the two frames so that 
an image within the second patch matches that within the reference image; 

estimating correspondent relationships among the-pixels within the second patch 
of the other frame and the-pixels within the reference patch of the reference frame, based on the 
second patch after the movement and/or deformation thereof and the reference patch; 

obtaining a coordinate converted frame, by coordinate converting the image 
within the second patch into a coordinate space of the reference frame, based on the 
correspondent relationships; 

calculating a first correlative value, which represents the-adegree of correlation 
between the images within the reference patch of the reference frame and the coordinate 
converted frame, for each of the rectangular regions; 

increasing the number of said at least one rectangular regions within the reference 
patch and the second patch in a stepwise manner by one step, estimating the afor e m e ntioned 
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correspondent relationships, obtaining the coordinate converted frames, and calculating the 
correlative value, thereby obtaining a second correlative value; 

calculating the variation between the first and second correlative values for each 
of the rectangular regions; 

increasing the number of said at least one rectangular regions within the reference 
patch and the second patch in a stepwise manner an additional step, estimating the 
afor e mention e d correspondent relationships, obtaining the coordinate converted frames, 
calculating the first and second correlative values, and calculating the variation between the first 
and second correlative values of consecutive steps, until the variation or a representative 
variation of all of said at least one the-rectangular regions, or of said at least one rectangular 
regions at predetermined positions from among all of said at least onet he rectangular regions, 
other than those for which the variation is greater than or equal to a predetermined value, or of a 
predetermined number of said at least one rectangular regions from among all of said at least 
onethe rectangular regions, other than those for which the variation is greater than or equal to a 
predetermined value, becomes less than or equal to a predetermined threshold value; and 

generating a synthesized frame having a higher resolution than either of the two 
frames, by administering interpolation calculations on the images within the second patch and 
the reference patch for each region corresponding to said at least onet he rectangular regions, 
based on the correspondent relationships, which were estimated among the pixels of either of the 
reference patches and the second patches having the number of said at least one rectangular 
regions therein that yielded the variation or the representative variation less than or equal to the 
threshold value. 
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33. (currently amended): A method for synthesizing moving images, comprising the 
steps of: 

sampling at least three consecutive frames from a moving image; 

providing a reference patch having at least one rectangular region therein, in a 
first frame from among the at least three sampled frames, which is designated as a reference 
frame; 

providing a second patch, which is the same as the reference patch, in another of 
the plurality of sampled frames; 

moving and/or deforming the second patch in the other frame so that an image 
within the second patch matches that within the reference image; 

estimating correspondent relationships among the-pixels within the second patch 
of the other frame and the-pixels within the reference patch of the reference frame, based on the 
second patch after the movement and/or deformation thereof and the reference patch; 

obtaining a coordinate converted frame, by coordinate converting the image 
within the second patch into a coordinate space of the reference frame, based on the 
correspondent relationships; 

calculating a first correlative value, which represents the-a^degree of correlation 
between the images within the reference patch of the reference frame and the coordinate 
converted frame; 

increasing the number of said at least one rectangular regions within the reference 
patch and the second patch in a stepwise manner by one step, estimating the afor e m e ntion e d 
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correspondent relationships, obtaining the coordinate converted frames, and calculating the 
correlative value, thereby obtaining a second correlative value; 

calculating the variation between the first and second correlative values; 

increasing the number of said at least one rectangular regions within the reference 
patch and the second patch in a stepwise manner an additional step, estimating the 
afor e mentioned correspondent relationships, obtaining the coordinate converted frames, 
calculating the first and second correlative values, and calculating the variation between the first 
and second correlative values of consecutive steps, until the variation becomes less than or equal 
to a predetermined threshold value; 

obtaining an intermediate synthesized frame having a higher resolution than either 
the reference frame of the other frame, by administering interpolation calculations on the images 
within the second patch and the reference patch, based on the correspondent relationships, which 
were estimated among the pixels of the reference patch and the second patch having the number 
of said at least one rectangular regions therein that yielded the variation less than or equal to the 
threshold value; 

obtaining a plurality of intermediate synthesized frames by estimating the 
afor e m e ntion e d correspondent relationships, obtaining the coordinate converted frames, 
calculating the plurality of correlative values, and calculating the variations among the plurality 
of the correlative values, for all of the plurality of sampled frames; and 

generating a synthesized frame by synthesizing the plurality of intermediate 
synthesized frames. 
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34. (currently amended): A method for synthesizing moving images, comprising the 
steps of: 

sampling at least three consecutive frames from a moving image; 

providing a reference patch having at least one rectangular region therein, in a 
first frame from among the at least three sampled frames, which is designated as a reference 
frame; 

providing a second patch, which is the same as the reference patch, in another of 
the plurality of sampled frames; 

moving and/or deforming the second patch in the other frame so that an image 
within the second patch matches that within the reference image; 

estimating correspondent relationships among tha-pixels within the second patch 
of the other frame and the-pixels within the reference patch of the reference frame, based on the 
second patch after the movement and/or deformation thereof and the reference patch; 

obtaining a coordinate converted frame, by coordinate converting the image 
within the second patch into a coordinate space of the reference frame, based on the 
correspondent relationships; 

calculating a first correlative value, which represents the-a_degree of correlation 
between the images within the reference patch of the reference frame and the coordinate 
converted frame, for each of the rectangular regions; 

increasing the number of said at least one rectangular regions within the reference 
patch and the second patch in a stepwise manner by one step, estimating the afor e m e ntion e d 
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correspondent relationships, obtaining the coordinate converted frames, and calculating the 
correlative value, thereby obtaining a second correlative value; 

calculating the variation between the first and second correlative values for each 
of the rectangular regions; 

increasing the number of said at least one rectangular regions within the reference patch 
and the second patch in a stepwise manner an additional step, estimating the afor e m e ntion e d 
correspondent relationships, obtaining the coordinate converted frames, calculating the first and 
second correlative values, and calculating the variation between the first and second correlative 
values of consecutive steps, until the variation or a representative variation of all of said at least 
onethe rectangular regions, or of said at least one the rectangular regions at predetermined 
positions from among all of said at least one the rectangular regions, other than those for which 
the variation is greater than or equal to a predetermined value, or of a predetermined number of 
said at least one rectangular regions from among all of said at least one the rectangular regions, 
other than those for which the variation is greater than or equal to a predetermined value, 
becomes less than or equal to a predetermined threshold value; 

obtaining an intermediate synthesized frame having a higher resolution than either 
the reference frame of the other frame, by administering interpolation calculations on the images 
within the second patch and the reference patch for each region corresponding to said at least one 
the-rectangular regions, based on the correspondent relationships, which were estimated among 
the pixels of the reference patch and the second patch having the number of said at least one 
rectangular regions therein that yielded the variation or the representative variation less than or 
equal to the threshold value; 
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obtaining a plurality of intermediate synthesized frames by estimating the 
afor e m e ntion e d correspondent relationships, obtaining the coordinate converted frames, 
calculating the plurality of correlative values, and calculating the variations among the plurality 
of the correlative values, for all of the plurality of sampled frames; and 

generating a synthesized frame by synthesizing the plurality of intermediate 
synthesized frames. 

35. (currently amended): A moving image synthesizing apparatus, comprising: 

a sampling means, for sampling two consecutive frames from a moving image; 
a correspondent relationship estimating means, for: providing a reference patch 
having at least one rectangular region therein, in one of the two frames, which is designated as a 
reference frame; providing a second patch, which is the same as the reference patch, in the other 
of the two frames; moving and/or deforming the second patch in the other of the two frames so 
that an image within the second patch matches that within the reference image; and estimating 
correspondent relationships among the-pixels within the second patch of the other frame and the 
pixels within the reference patch of the reference frame, based on the second patch after the 
movement and/or deformation thereof and the reference patch; 

a coordinate converting means, for obtaining a coordinate converted frame, by 
coordinate converting the image within the second patch into a coordinate space of the reference 
frame, based on the correspondent relationships; 
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a correlative value calculating means, for calculating a first correlative value, 
which represents the-adegree of correlation between the images within the reference patch of the 
reference frame and the coordinate converted frame; 

a control means, for controlling the correspondent relationship estimating means, 
the coordinate converting means, and the correlative value calculating means, to increase the 
number of said at least one rectangular regions within the reference patch and the second patch in 
a stepwise manner by one step, to estimate the afor e m e ntioned correspondent relationships, to 
obtain the coordinate converted frames, and to calculate the correlative value, thereby obtaining 
a second correlative value; 

a variation calculating means, for calculating the variation between the first and 
second correlative values; 

a comparing means, for controlling the correspondent relationship estimating 
means, the coordinate converting means, the correlative relationship calculating means, and the 
variation calculating means, to increase the number of said at least one rectangular regions 
within the reference patch and the second patch in a stepwise manner an additional step, to 
estimate the aforem e ntion e d correspondent relationships, to obtain the coordinate converted 
frames, to calculate the first and second correlative values, and to calculate the variation between 
the first and second correlative values of consecutive steps, until the variation becomes less than 
or equal to a predetermined threshold value; and 

a synthesizing means, for generating a synthesized frame having a higher 
resolution than either of the two frames, by administering interpolation calculations on the 
images within the second patch and the reference patch, based on the correspondent 
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relationships, which were estimated among the pixels of either of the reference patches and the 
second patches having the number of said at least one rectanRular regions therein that yielded the 
variation less than or equal to the threshold value. 

36. (currently amended): A moving image synthesizing apparatus, comprising: 

a sampling means, for sampling two consecutive frames from a moving image; 

a correspondent relationship estimating means, for: providing a reference patch 
having at least one rectangular region therein, in one of the two frames, which is designated as a 
reference frame; providing a second patch, which is the same as the reference patch, in the other 
of the two frames; moving and/or deforming the second patch in the other of the two frames so 
that an image within the second patch matches that within the reference image; and estimating 
correspondent relationships among the-pixels within the second patch of the other frame and the 
pixels within the reference patch of the reference frame, based on the second patch after the 
movement and/or deformation thereof and the reference patch; 

a coordinate converting means, for obtaining a coordinate converted frame, by 
coordinate converting the image within the second patch into a coordinate space of the reference 
frame, based on the correspondent relationships; 

a correlative value calculating means, for calculating a first correlative value, 
which represents the-adegree of correlation between the images within the reference patch of the 
reference frame and the coordinate converted frame, for each of the rectangular regions; 

a control means, for controlling the correspondent relationship estimating means, 
the coordinate converting means, and the correlative value calculating means, to increase the 

46 



AMENDMENT UNDER 37 C.F.R. §1.111 Attorney Docket No.: Q79263 

Application No.: 10/750,461 

number of said at least one rectangular regions within the reference patch and the second patch in 
a stepwise manner by one step, to estimate the aforem e ntion e d correspondent relationships, to 
obtain the coordinate converted frames, and to calculate the correlative value, thereby obtaining 
a second correlative value; 

a variation calculating means, for calculating the variation between the first and 
second correlative values for each of the rectangular regions; 

a comparing means, for controlling the correspondent relationship estimating 
means, the coordinate converting means, the correlative relationship calculating means, and the 
variation calculating means, to increase the number of said at least one rectangular regions 
within the reference patch and the second patch in a stepwise manner an additional step, to 
estimate the afor e m e ntion e d correspondent relationships, to obtain the coordinate converted 
frames, to calculate the first and second correlative values, and to calculate the variation between 
the first and second correlative values of consecutive steps, until the variation or a representative 
variation of all of the- said at least one rectangular regions, or of said at least one the rectangular 
regions at predetermined positions from among all of the said at least one rectangular regions, 
other than those for which the variation is greater than or equal to a predetermined value, or of a 
predetermined number of said at least one rectangular regions from among all of thesaid at least 
one rectangular regions, other than those for which the variation is greater than or equal to a 
predetermined value, becomes less than or equal to a predetermined threshold value; and 

a synthesizing means, for generating a synthesized frame having a higher 
resolution than either of the two frames, by administering interpolation calculations on the 
images within the second patch and the reference patch for each region corresponding to said at 
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least one the rectangular regions, based on the correspondent relationships, which were estimated 
among the pixels of either of the reference patches and the second patches having the number of 
said at least one rectangular regions therein that yielded the variation or the representative 
variation less than or equal to the threshold value. 

37. (original): A moving image synthesizing apparatus as defined in claim 35, wherein: 

the control means sets the predetermined threshold value to be larger, according 
to the increase in the number of rectangular regions, when the number of rectangular regions 
exceeds a predetermined value. 

38. (original): A moving image synthesizing apparatus as defined in claim 36, wherein: 

the control means sets the predetermined threshold value to be larger, according 
to the increase in the number of rectangular regions, when the number of rectangular regions 
exceeds a predetermined value. 

39. (original): A moving image synthesizing apparatus as defined in claim 35, wherein: 

the correspondent relationship estimating means, the coordinate converting 
means, the correlative value calculating means, the control means, the variation calculating 
means, the comparing means, and the synthesizing means employ at least one component that 
constitutes the frames during the estimation of the correspondent relationships, the obtainment of 
the coordinate converted frames, the calculation of the correlative values, the calculation of the 
variations, and the generation of the synthesized frame. 
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40. (original): A moving image synthesizing apparatus as defined in claim 36, wherein: 

the correspondent relationship estimating means, the coordinate converting 
means, the correlative value calculating means, the control means, the variation calculating 
means, the comparing means, and the synthesizing means employ at least one component that 
constitutes the frames during the estimation of the correspondent relationships, the obtainment of 
the coordinate converted frames, the calculation of the correlative values, the calculation of the 
variations, and the generation of the synthesized frame. 

41. (currently amended): A moving image synthesizing apparatus, comprising: 

a sampling means, for sampling at least three consecutive frames from a moving 

image; 

a correspondent relationship estimating means, for: providing a reference patch 
having at least one rectangular region therein, in a first frame from among the at least three 
sampled frames, which is designated as a reference frame; providing a second patch, which is the 
same as the reference patch, in another of the plurality of sampled frames; moving and/or 
deforming the second patch in the other frame so that an image within the second patch matches 
that within the reference image; and estimating correspondent relationships among the-pixels 
within the second patch of the other frame and the-pixels within the reference patch of the 
reference frame, based on the second patch after the movement and/or deformation thereof and 
the reference patch; 
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a coordinate converting means, for obtaining a coordinate converted frame, by 
coordinate converting the image within the second patch into a coordinate space of the reference 
frame, based on the correspondent relationships; 

a correlative value calculating means, for calculating a first correlative value, 
which represents the-a_degree of correlation between the images within the reference patch of the 
reference frame and the coordinate converted frame; 

a control means, for controlling the correspondent relationship estimating means, 
the coordinate converting means, and the correlative value calculating means, to increase the-a 
number of said at least one rectangular regions within the reference patch and the second patch in 
a stepwise manner by one step, to estimate the aforem e ntion e d correspondent relationships, to 
obtain the coordinate converted frames, and to calculate the correlative value, thereby obtaining 
a second correlative value; 

a variation calculating means, for calculating the variation between the first and 
second correlative values; 

a comparing means, for controlling the correspondent relationship estimating 
means, the coordinate converting means, the correlative value calculating means, and the 
variation calculating means, to increase the number of said at least one rectangular regions 
within the reference patch and the second patch in a stepwise manner an additional step, to 
estimate the afor e mention e d correspondent relationships, to obtain the coordinate converted 
frames, to calculate the first and second correlative values, and to calculate the variation between 
the first and second correlative values of consecutive steps, until the variation becomes less than 
or equal to a predetermined threshold value; and 
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a synthesizing means, for: obtaining an intermediate synthesized frame having a 
higher resolution than either the reference frame of the other frame, by administering 
interpolation calculations on the images within the second patch and the reference patch, based 
on the correspondent relationships, which were estimated among the pixels of the reference patch 
and the second patch having the number of said at least one rectangular regions therein that 
yielded the variation less than or equal to the threshold value; obtaining a plurality of 
intermediate synthesized frames by estimating the afor e mentioned correspondent relationships, 
obtaining the coordinate converted frames, calculating the plurality of correlative values, and 
calculating the variations among the plurality of the correlative values, for all of the plurality of 
sampled frames; and generating a synthesized frame by synthesizing the plurality of intermediate 
synthesized frames. 

42. (currently amended): A moving image synthesizing apparatus, comprising: 

a sampling means, for sampling at least three consecutive frames from a moving 

image; 

a correspondent relationship estimating means, for: providing a reference patch 
having at least one rectangular region therein, in a first frame from among the at least three 
sampled frames, which is designated as a reference frame; providing a second patch, which is the 
same as the reference patch, in another of the plurality of sampled frames; moving and/or 
deforming the second patch in the other frame so that an image within the second patch matches 
that within the reference image; and estimating correspondent relationships among the-pixels 
within the second patch of the other frame and the-pixels within the reference patch of the 
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reference frame, based on the second patch after the movement and/or deformation thereof and 
the reference patch; 

a coordinate converting means, for obtaining a coordinate converted frame, by 
coordinate converting the image within the second patch into a coordinate space of the reference 
frame, based on the correspondent relationships; 

a correlative value calculating means, for calculating a first correlative value, 
which represents the-a_degree of correlation between the images within the reference patch of the 
reference frame and the coordinate converted frame, for each of the rectangular regions; 

a control means, for controlling the correspondent relationship estimating means, 
the coordinate converting means, and the correlative value calculating means, to increase the-a 
number of rectangular regions within the reference patch and the second patch in a stepwise 
manner by one step, to estimate the afor e m e ntion e d correspondent relationships, to obtain the 
coordinate converted frames, and to calculate the correlative value, thereby obtaining a second 
correlative value; 

a variation calculating means, for calculating the variation between the first and 
second correlative values for each of the rectangular regions; 

a comparing means, for controlling the correspondent relationship estimating means, the 
coordinate converting means, the correlative value calculating means, and the variation 
calculating means, to increase the number o f said at least one rectangular regions within the 
reference patch and the second patch in a stepwise manner an additional step, to estimate the 
aforemention e d correspondent relationships, to obtain the coordinate converted frames, to 
calculate the first and second correlative values, and to calculate the variation between the first 
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and second correlative values of consecutive steps, until the variation of all of said at least one the 
rectangular regions, or of said at least one the rectangular regions at predetermined positions from 
among all of said at least one the rectangular regions, other than those for which the variation is 
greater than or equal to a predetermined value, or of a predetermined number of said at least one 
rectangular regions from among all of said at least one the rectangular regions, other than those 
for which the variation is greater than or equal to a predetermined value, becomes less than or 
equal to a predetermined threshold value; and 

a synthesizing means, for: obtaining an intermediate synthesized frame having a 
higher resolution than either the reference frame of the other frame, by administering 
interpolation calculations on the images within the second patch and the reference patch, based 
on the correspondent relationships, which were estimated among the pixels of the reference patch 
and the second patch having the number o f said at least one rectangular regions therein that 
yielded the variation or the representative variation less than or equal to the threshold value; 
obtaining a plurality of intermediate synthesized frames by estimating the aforem e ntion e d 
correspondent relationships, obtaining the coordinate converted frames, calculating the plurality 
of correlative values, and calculating the variations among the plurality of the correlative values, 
for all of the plurality of sampled frames; and generating a synthesized frame by synthesizing the 
plurality of intermediate synthesized frames. 

43. (original): A moving image synthesizing apparatus as defined in claim 41, wherein: 
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the control means sets the predetermined threshold value to be larger, according 
to the increase in the number of rectangular regions, when the number of rectangular regions 
exceeds a predetermined value. 

44. (original): A moving image synthesizing apparatus as defined in claim 42, wherein: 

the control means sets the predetermined threshold value to be larger, according 
to the increase in the number of rectangular regions, when the number of rectangular regions 
exceeds a predetermined value. 

45. (original): A moving image synthesizing apparatus as defined in claim 41, wherein: 

the correspondent relationship estimating means, the coordinate converting 
means, the correlative value calculating means, the control means, the variation calculating 
means, the comparing means, and the synthesizing means employ at least one component that 
constitutes the frames during the estimation of the correspondent relationships, the obtainment of 
the coordinate converted frames, the calculation of the correlative values, the calculation of the 
variations, the generation of the intermediate synthesized frames, and the generation of the 
synthesized frame. 

46. (original): A moving image synthesizing apparatus as defined in claim 42, wherein: 

the correspondent relationship estimating means, the coordinate converting 
means, the correlative value calculating means, the control means, the variation calculating 
means, the comparing means, and the synthesizing means employ at least one component that 
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constitutes the frames during the estimation of the correspondent relationships, the obtainment of 
the coordinate converted frames, the calculation of the correlative values, the calculation of the 
variations, the generation of the intermediate synthesized frames, and the generation of the 
synthesized frame. 

47. (currently amended): A recordinR medium having embodied thereon Aa program 
that causes a computer to execute a method for synthesizing moving images, comprising the 
procedures of: 

sampling two consecutive frames from a moving image; 

providing a reference patch having at least one rectangular region therein, in one 
of the two frames, which is designated as a reference frame; 

providing a second patch, which is the same as the reference patch, in the other of 
the two frames; 

moving and/or deforming the second patch in the other of the two frames so that 
an image within the second patch matches that within the reference image; 

estimating correspondent relationships among the-pixels within the second patch 
of the other frame and the-pixels within the reference patch of the reference frame, based on the 
second patch after the movement and/or deformation thereof and the reference patch; 

obtaining a coordinate converted frame, by coordinate converting the image 
within the second patch into a coordinate space of the reference frame, based on the 
correspondent relationships; 
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calculating a first correlative value, which represents the-adegree of correlation 
between the images within the reference patch of the reference frame and the coordinate 
converted frame; 

increasing the number of said at least one rectangular regions within the reference 
patch and the second patch in a stepwise manner by one step, estimating the afor e m e ntion e d 
correspondent relationships, obtaining the coordinate converted frames, and calculating the 
correlative value, thereby obtaining a second correlative value; 

calculating the variation between the first and second correlative values; 
increasing the number of said at least one rectangular regions within the reference patch 
and the second patch in a stepwise manner an additional step, estimating the afor e m e ntion e d 
correspondent relationships, obtaining the coordinate converted frames, calculating the first and 
second correlative values, and calculating the variation between the first and second correlative 
values of consecutive steps, until the variation becomes less than or equal to a predetermined 
threshold value; and 

generating a synthesized frame having a higher resolution than either of the two 
frames, by administering interpolation calculations on the images within the second patch and 
the reference patch, based on the correspondent relationships, which were estimated among the 
pixels of either of the reference patches and the second patches having the number of said at least 
one rectangular regions therein that yielded the variation less than or equal to the threshold value. 
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48. (currently amended): A recording medium having embodied thereon Aa program 
that causes a computer to execute a method for synthesizing moving images, comprising the 
procedures of: 

sampling two consecutive frames from a moving image; 

providing a reference patch having at least one rectangular region therein, in one 
of the two frames, which is designated as a reference frame; 

providing a second patch, which is the same as the reference patch, in the other of 
the two frames; 

moving and/or deforming the second patch in the other of the two frames so that 
an image within the second patch matches that within the reference image; 

estimating correspondent relationships among the-pixels within the second patch 
of the other frame and the-pixels within the reference patch of the reference frame, based on the 
second patch after the movement and/or deformation thereof and the reference patch; 

obtaining a coordinate converted frame, by coordinate converting the image 
within the second patch into a coordinate space of the reference frame, based on the 
correspondent relationships; 

calculating a first correlative value, which represents the-ajiegree of correlation 
between the images within the reference patch of the reference frame and the coordinate 
converted frame, for each of said at least one the rectangular regions; 

increasing the number o f said at least one rectangular regions within the reference 
patch and the second patch in a stepwise manner by one step, estimating the afor e m e ntion e d 
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correspondent relationships, obtaining the coordinate converted frames, and calculating the 
correlative value, thereby obtaining a second correlative value; 

calculating the variation between the first and second correlative values for each 
of said at least onet he rectangular regions; 

increasing the number of said at least one rectangular regions within the reference 
patch and the second patch in a stepwise manner an additional step, estimating the 
afor e m e ntion e d correspondent relationships, obtaining the coordinate converted frames, 
calculating the first and second correlative values, and calculating the variation between the first 
and second correlative values of consecutive steps, until the variation or a representative 
variation of all of said at least one the rectangular regions, or of said at least one the rectangular 
regions at predetermined positions from among all of said at least one the rectangular regions, 
other than those for which the variation is greater than or equal to a predetermined value, or of a 
predetermined number of said at least one rectangular regions from among all of said at least 
onethe rectangular regions, other than those for which the variation is greater than or equal to a 
predetermined value, becomes less than or equal to a predetermined threshold value; and 

generating a synthesized frame having a higher resolution than either of the two 
frames, by administering interpolation calculations on the images within the second patch and 
the reference patch for each region corresponding to the rectangular regions, based on the 
correspondent relationships, which were estimated among the pixels of either of the reference 
patches and the second patches having the number of said at least one rectangular regions therein 
that yielded the variation or the representative variation less than or equal to the threshold value. 



58 



AMENDMENT UNDER 37 C.F.R. §1.111 Attorney Docket No.: Q79263 

Application No.: 10/750,461 

49. (currently amended): A recording medium having embodied thereon Aa program 
that causes a computer to execute a method for synthesizing moving images, comprising the 
procedures of: 

sampling at least three consecutive frames from a moving image; 

providing a reference patch having at least one rectangular region therein, in a 
first frame from among the at least three sampled frames, which is designated as a reference 
frame; 

providing a second patch, which is the same as the reference patch, in another of 
the plurality of sampled frames; 

moving and/or deforming the second patch in the other frame so that an image 
within the second patch matches that within the reference images 

estimating correspondent relationships among the-pixels within the second patch 
of the other frame and the-pixels within the reference patch of the reference frame, based on the 
second patch after the movement and/or deformation thereof and the reference patch; 

obtaining a coordinate converted frame, by coordinate converting the image 
within the second patch into a coordinate space of the reference frame, based on the 
correspondent relationships; 

calculating a first correlative value, which represents the-adegree of correlation 
between the images within the reference patch of the reference frame and the coordinate 
converted frame; 

increasing the number of said at least one rectangular regions within the reference 
patch and the second patch in a stepwise manner by one step, estimating the afor e m e ntion e d 
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correspondent relationships, obtaining the coordinate converted frames, and calculating the 
correlative value, thereby obtaining a second correlative value; 

calculating the variation between the first and second correlative values; 

increasing the number of said at least one rectangular regions within the reference 
patch and the second patch in a stepwise manner an additional step, estimating the 
afor e m e ntion e d correspondent relationships, obtaining the coordinate converted frames, 
calculating the first and second correlative values, and calculating the variation between the first 
and second correlative values of consecutive steps, until the variation becomes less than or equal 
to a predetermined threshold value; 

obtaining an intermediate synthesized frame having a higher resolution than either 
the reference frame of the other frame, by administering interpolation calculations on the images 
within the second patch and the reference patch, based on the correspondent relationships, which 
were estimated among the pixels of the reference patch and the second patch having the number 
of said at least one rectangular regions therein that yielded the variation less than or equal to the 
threshold value; 

obtaining a plurality of intermediate synthesized frames by estimating the 
afor e m e ntion e d correspondent relationships, obtaining the coordinate converted frames, 
calculating the plurality of correlative values, and calculating the variations among the plurality 
of the correlative values, for all of the plurality of sampled frames; and 

generating a synthesized frame by synthesizing the plurality of intermediate 
synthesized frames. 
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50. (currently amended): A recording medium having embodied thereon Aa program 
that causes a computer to execute a method for synthesizing moving images, comprising the 
procedures of: 

sampling at least three consecutive frames from a moving image; 

providing a reference patch having at least one rectangular region therein, in a 
first frame from among the at least three sampled frames, which is designated as a reference 
frame; 

providing a second patch, which is the same as the reference patch, in another of 
the plurality of sampled frames; 

moving and/or deforming the second patch in the other frame so that an image 
within the second patch matches that within the reference image; 

estimating correspondent relationships among the-pixels within the second patch 
of the other frame and the-pixels within the reference patch of the reference frame, based on the 
second patch after the movement and/or deformation thereof and the reference patch; 

obtaining a coordinate converted frame, by coordinate converting the image 
within the second patch into a coordinate space of the reference frame, based on the 
correspondent relationships; 

calculating a first correlative value, which represents the-a_degree of correlation 
between the images within the reference patch of the reference frame and the coordinate 
converted frame, for each of the rectangular regions; 

increasing the number of said at least one rectangular regions within the reference 
patch and the second patch in a stepwise manner by one step, estimating the afor e m e ntion e d 
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correspondent relationships, obtaining the coordinate converted frames, and calculating the 
correlative value, thereby obtaining a second correlative value; 

calculating the variation between the first and second correlative values for each 
of the rectangular regions; 

increasing the number of said at least one rectangular regions within the reference 
patch and the second patch in a stepwise manner an additional step, estimating the 
aforem e ntion e d correspondent relationships, obtaining the coordinate converted frames, 
calculating the first and second correlative values, and calculating the variation between the first 
and second correlative values of consecutive steps, until the variation or a representative 
variation of all of said at least one the rectangular regions, or of said at least one the rectangular 
regions at predetermined positions from among all of said at least one the rectangular regions, 
other than those for which the variation is greater than or equal to a predetermined value, or of a 
predetermined number of said at least one rectangular regions from among all of said at least 
onethe rectangular regions, other than those for which the variation is greater than or equal to a 
predetermined value, becomes less than or equal to a predetermined threshold value; 

obtaining an intermediate synthesized frame having a higher resolution than either 
the reference frame of the other frame, by administering interpolation calculations on the images 
within the second patch and the reference patch for each region corresponding to said at least 
onethe rectangular regions, based on the correspondent relationships, which were estimated 
among the pixels of the reference patch and the second patch having the number of said at least 
one rectangular regions therein that yielded the variation or the representative variation less than 
or equal to the threshold value; 
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obtaining a plurality of intermediate synthesized frames by estimating the 
afor e m e ntion e d correspondent relationships, obtaining the coordinate converted frames, 
calculating the plurality of correlative values, and calculating the variations among the plurality 
of the correlative values, for all of the plurality of sampled frames; and 

generating a synthesized frame by synthesizing the plurality of intermediate 
synthesized frames. 

51-60. (canceled). 
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